3.3 Taylor’s Theorem

When we introduced the derivative in Chapter 2, we saw that the linear approximation of a function played an essential role for a geometric reason—finding the equation of a tangent plane—as well as an analytic reason—finding approximate values of functions. Taylor’s theorem deals with the important issue of finding quadratic and higher-order approximations.

Taylor’s theorem is a central tool for finding accurate numerical approximations of functions, and as such plays an important role in many areas of applied and computational mathematics. We shall use it in the next section to develop the second derivative test for maxima and minima of functions of several variables.

The strategy used to prove Taylor’s theorem is to reduce it to the one-variable case by probing a function of many variables along lines of the form \({\bf l}(t) = {\bf x}_{0}+t{\bf h}\) emanating from a point \({\bf x}_{0}\) and heading in the direction \({\bf h}\). Thus, it will be useful for us to begin by reviewing Taylor’s theorem from one-variable calculus.

Single-Variable Taylor Theorem

When recalling a theorem from an earlier course, it is helpful to ask these basic questions: What is the main point of the theorem? What are the key ideas in the proof? Can I understand the result better the second time around?

The main point of the single-variable Taylor theorem is to find approximations of a function near a given point that are accurate to a higher order than the linear approximation. The key idea in the proof is to use the fundamental theorem of calculus, followed by integration by parts. In fact, just by recalling these basic ideas, we can reconstruct the entire proof. Thinking in this way will help organize all the pieces that need to come together to develop a mastery of Taylor approximations of functions of one and several variables.

For a smooth function \(f\): \({\mathbb R} \to {\mathbb R}\) of one variable, Taylor’s theorem asserts that: \begin{equation*} f(x_0\,{+}\,h) = f(x_0)\,{+}\,f^\prime (x_0)\,{\cdot}\,h + \frac{f^{\prime\prime}(x_0)}{2}h^2+\cdots+\frac{f^{(k)}(x_0)}{k!}h^k+R_k(x_0,h),\tag{1} \end{equation*} where \[ R_k(x_0,h)=\int_{x_0}^{x_0+h}\frac{(x_0+h-\tau)^k}{k!}f^{k+1}(\tau) \ d\tau \] is the remainder. For small \(h\), this remainder is small to order \(k\) in the sense that \begin{equation*} \lim_{h\to 0}\frac{R_k(x_0,h)}{h^k}=0.\tag{2} \end{equation*} In other words, \(R_{k}(x_{0}\), \(h)\) is small compared to the already small quantity \(h^{k}\).

The preceding is the formal statement of Taylor’s theorem. What about the proof? As promised, we begin with the fundamental theorem of calculus, written in the form: \[ f(x_0 + h)=f(x_0)+\int_{x_0}^{x_0+h}f^\prime(\tau)\ d\tau. \] Next, we write \(d\tau =-d(x_{0}+h - \tau)\) and integrate partsfootnote # to give: \[ f(x_0+h)=f(x_0)+f^\prime (x_0)h +\int_{x_0}^{x_0+h}f^{\prime\prime}(\tau)(x_0+h-\tau)\ d\tau, \] which is the first-order Taylor formula. Integrating by parts again: \begin{eqnarray*} && \int_{x_0}^{x_0+h} f^{\prime\prime}(\tau)(x_0+h-\tau)\ d\tau\\ &&\quad= -\frac{1}{2}\int_{x_0}^{x_0+h}f^{\prime\prime}(\tau)\ d(x_0+h-\tau)^2\\ &&\quad= \frac{1}{2}f^{\prime\prime}(x_0)h^2+\frac{1}{2}\int_{x_0}^{x_0+h}f^{\prime\prime\prime}(\tau)(x_0+h-\tau)^2\ d\tau, \end{eqnarray*} which, when substituted into the preceding formula, gives the second-order Taylor formula: \begin{eqnarray*} f(x_0+h) &=& f(x_0)+f^\prime(x_0)h+\frac{1}{2}f^{\prime\prime}(x_0)h^2 +\frac{1}{2}\int_{x_0}^{x_0+h}f^{\prime\prime\prime}(\tau)(x_0+h-\tau)^2\ d\tau. \end{eqnarray*} This is Taylor’s theorem for \(k = 2\).

Taylor’s theorem for general \(k\) proceeds by repeated integration by parts. The statement (2) that \(R_{k}(x_{0}\), \(h)/h^{k} \to 0\) as \(h \to 0\) is seen as follows. For \(\tau\) in the interval [\(x_{0}\), \(x_{0}+h\)], we have \(\vert x_{0}+h- \tau \vert \le \vert h\vert \), and \(f^{k + 1}(\tau)\), being continuous, is bounded; say, \(\vert f^{k + 1}(\tau )\vert \le M\). Then: \[ |R_k(x_0,h)|=\bigg|\int_{x_0}^{x_0+h}\frac{(x_0+h-\tau)^k}{k!}f^{k+1}(\tau)\ d\tau\bigg|\,{\le}\, \frac{|h|^{k+1}}{k!}M \] and, in particular, \(\vert R_{k}(x_{0}\), \(h){/}h^{k}\vert \le\vert h\vert \,{M/k!}\,\to 0\) as \(h \to 0\).

Taylor’s Theorem for Many Variables

Our next goal in this section is to prove an analogous theorem that is valid for functions of several variables. We already know a first-order version; that is, when \(k=1\). Indeed, if \(f{:}\,\, {\mathbb R}^n\rightarrow {\mathbb R}\) is differentiable at \({\bf x}_0\) and we define \[ R_1({\bf x}_0, {\bf h}) = f({\bf x}_{0} + {\bf h}) - f({\bf x}_0 ) - [{\bf D}\! f({\bf x}_0)] ({\bf h}), \] so that \[ f({\bf x}_0+{\bf h}) = f({\bf x}_0) + [{\bf D}\! f({\bf x}_0)] ({\bf h}) + R_1({\bf x}_0, {\bf h}), \] then by the definition of differentiability, \[ \frac{|R_1({\bf x}_0, {\bf h})|} {\| {\bf h}\|} \to 0 \quad \hbox{as}\quad {\bf h} \to 0; \] that is, \(R_1({\bf x}_0, {\bf h})\) vanishes to first order at \({\bf x}_0\). In summary, we have:

The second-order version is as follows:

Notice that this result can be written in matrix form as \begin{eqnarray*} f({\bf x}_0+{\bf h}) &=& f({\bf x}_0)+\bigg[\frac{\partial f}{\partial x_1},\ldots,\frac{\partial f}{\partial x_n}\bigg]\left[ \begin{array}{c} h_1\\ \vdots\\ h_n \end{array}\right]\\[2pt] &&+\,\frac{1}{2}[h_1,\ldots,h_n]\left[ \begin{array}{c@{\quad}c@{\quad}c@{\quad}c} \\[-9pt] \displaystyle\frac{\partial^2\!f}{\partial x_1\,\partial x_1} & \displaystyle\frac{\partial^2\!f}{\partial x_1\,\partial x_2} & \cdots & \displaystyle\frac{\partial^2\!f}{\partial x_1\,\partial x_n}\\[13.5pt] \displaystyle\frac{\partial^2\!f}{\partial x_2\,\partial x_1} & \displaystyle\frac{\partial^2\!f}{\partial x_2\,\partial x_2} & \cdots & \displaystyle\frac{\partial^2\!f}{\partial x_2\,\partial x_n}\\[8pt] \vdots\\[4pt] \displaystyle\frac{\partial^2\!f}{\partial x_n\,\partial x_1} & \displaystyle\frac{\partial^2\!f}{\partial x_n\,\partial x_2} & \cdots & \displaystyle\frac{\partial^2\!f}{\partial x_n\,\partial x_n}\\[9pt] \end{array}\right]\left[ \begin{array}{c} h_1\\[6pt] h_2\\[1pt] \vdots\\[4pt] h_n \end{array}\right],\\[2pt] &&+\,R_2({\bf x}_0, {\bf h}), \end{eqnarray*} where the derivatives of \(f\) are evaluated at \({\bf x}_0\).

In the course of the proof of the Theorem 3, we shall obtain a useful explicit formula for the remainder, as in the single-variable theorem.

The formulas involving \({\bf c}_{\it ij}\) and \({\bf c}_{\it ijk}\) (called Lagrange’s form of the remainder) are obtained by making use of the second mean-value theorem for integrals. This states that \[ \int_a^b h(t) g(t) \,{\it dt} = h(c) \int_a^b g(t) \,{\it dt}, \] provided \(h\) and \(g\) are continuous and \(g \ge 0\) on \([a,b];\) here \(c\) is some number between \(a\) and \(b\).footnote # This is applied in formula (4) for the explicit form of the remainder with \(h(t) = (\partial^2\! f/\partial x_i \partial x_j)({\bf x}_0 + t{\bf h})\) and \(g(t) = 1-t\).

The third-order Taylor formula is \begin{eqnarray*} f({\bf x}_0 + {\bf h}) &=& f({\bf x}_0) + \sum_{i=1}^n\, h_i \frac{\partial f}{\partial x_i} ({\bf x}_0) + \frac12 \sum_{i,j=1}^n h_ih_j \frac{\partial^2\! f}{\partial x_i\,\partial x_j} ({\bf x}_0)\\[5pt] &&{+}\,\frac{1}{3!} \sum_{i,j,k=1}^n h_ih_jh_k \frac{\partial^3\! f}{\partial x_i\,\partial x_j\,\partial x_k} ({\bf x}_0) + R_3({\bf x}_0, {\bf h}), \end{eqnarray*} where \(R_3({\bf x}_0, {\bf h})/\|{\bf h}\|^3\to 0\) as \({\bf h}\to {\bf 0}\), and so on. The general formula can be proved by induction, using the method of proof already given.

In the case of functions of one variable, we can expand \(f(x)\) in an infinite power series, called the Taylor series: \[ f(x_0+h) = f(x_0) + f'(x_0)h + \frac{f”(x_0)h^2}{2} + \cdots+ \frac{f^{(k)} (x_0)h^k}{k!} +\cdots, \] provided we can show that \(R_k(x_0,h) \to 0\) as \(k \to \infty\). Similarly, for functions of several variables, the preceding terms are replaced by the corresponding ones involving partial derivatives, as we have seen in Theorem 3. Again, we can represent such a function by its Taylor series provided we can show that \(R_k \to 0\) as \(k \to \infty\). This point is examined further in Exercise 13.

The first-, second-, and third-order Taylor polynomials are also called the first-, second-, and third-order Taylor approximations to \(f\), since it is presumed that the remainder is small and gets smaller as the order of the Taylor polynomial increases.